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EXECUTIVE  SUMMARY 


The  Remote  Minehunting  System  (RMS)  is  a  mine  reconnaissance  system  for  the 
detection,  classification,  identification,  and  localization  of  bottom  and  moored  mine-like 
objects  in  shallow  and  moderately  deep  water.  In  December  2009  the  Navy  notified 
Congress  of  a  critical  Nunn-McCurdy  breach  in  both  the  Program  Acquisition  Unit  Cost 
(PAUC)  and  Average  Procurement  Unit  Cost  (APUC).  The  Navy  reported  that  the  PAUC 
had  risen  85.1  percent,  from  $12.1  million  in  the  October  2006  Acquisition  Program 
Baseline  (APB)  to  $22.4  million.  Over  the  same  time  span  the  APUC  rose  51.2  percent, 
from  $8.4  million  to  $12.7  million. 

The  analysis  revealed  three  major  reasons  for  the  cost  growth.  First,  the  Navy 
reduced  its  planned  procurement  quantity  from  106  to  52,  as  it  decided  to  limit  the  RMS 
to  the  Mine  Countenneasure  mission  on  the  Littoral  Combat  Ship  (LCS)  and  no  longer 
procure  the  system  for  the  Anti-Submarine  Warfare  mission  package.  The  quantity 
reduction  accounted  for  over  40  percent  of  the  reported  PAUC  cost  growth  and  18 
percent  of  the  APUC  cost  growth.  Second,  the  design  fell  far  short  of  its  reliability  and 
availability  requirements,  thus  requiring  additional  funding  to  improve  reliability  and 
correct  flaws,  and  delays  to  production  to  implement  the  changes.  The  associated 
development  cost  and  delays  increased  PAUC  and  APUC  by  22  percent  and  six  percent, 
respectively.  Third,  the  Acquisition  Program  Baseline  (APB)  cost  estimate  did  not 
include  some  cost  data  from  a  negotiated  contract.  The  cost  estimate  was  based  on  the 
average  price  of  only  the  recurring  hardware  production  for  the  Low  Rate  Initial 
Production  (LRIP)  units  and  did  not  include  hardware  and  software  engineering, 
integration  and  testing,  or  program  management  costs.  However,  the  contract  price  for 
those  units  includes  those  elements  and  is  significantly  higher.  Fixing  this  mistake 
increased  the  PAUC  by  14  percent  and  the  APUC  by  21  percent.  Table  S-l  apportions  the 
cost  growth  to  the  major  drivers.  The  paper  will  detail  the  rationale  and  calculations 
behind  each  of  the  major  cost  drivers. 

The  analysis  also  examined  the  root  causes  behind  the  three  proximate  causes.  Two 
root  causes  are  directly  related  to  their  proximate  causes  (procurement  reduction  and  a 
baseline  cost  error).  The  decision  to  reduce  the  procurement  quantities  appears  to  be 
exogenous  to  the  program  and  unrelated  to  the  reliability  issues.  The  baseline  cost 
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estimate  was  unrealistic,  since  the  original  official  estimate  did  not  include  some  cost 
data.  The  final  root  cause — failure  of  government  oversight — manifested  itself  in 
reliability  issues. 

Table  S-1.  2009  Nunn-McCurdy  Cost  Growth  Apportioned  among  the  Major  Drivers 


In  $M  BY 
FY  2006 

Current  Navy 

Original  APB  Estimate 

Delta 

%  Growth 

PAUC 

12.1  22.4 

+  10.3 

85.1% 

Qty  Reduction  &  Profile  Change 

+  5.17 

43% 

Quantity  Reduction 

+  4.24 

35% 

Reduced  Production  Rate 

+  0.93 

8% 

Reliability  Issues 

+  2.71 

22% 

Reliability  Growth  Program 

+  2.22 

18% 

Five-year  Gap 

+  0.49 

4% 

Unrealistic  unit  cost  estimation 

+  1.68 

14% 

Other 

+  0.74 

6% 

APUC 

8.4  12.7 

+  4.3 

51.2% 

Qty  Reduction  &  Profile  Change 

+  1.52 

18% 

Quantity  Reduction 

+  0.54 

6% 

Reduced  Production  Rate 

+  0.98 

12% 

Reliability  Issues:  Five-year  Gap 

+  0.51 

6% 

Unrealistic  unit  cost  estimation 

+  1.74 

21% 

Other 

+  0.53 

6% 
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I.  BACKGROUND 


The  Remote  Minehunting  System  (RMS)  is  a  system  for  the  detection, 
classification,  identification,  and  localization  of  bottom  and  moored  mines  in  shallow  and 
deep  water.  It  is  intended  to  keep  ships  and  sailors  out  of  a  minefield  and  will  be  installed 
on  the  Littoral  Combat  Ship  (LCS)  as  part  of  the  ship’s  Mine  Warfare  Mission  Package. 
In  December  2009,  the  Navy  notified  Congress  of  a  critical  Nunn-McCurdy  breach  in 
both  the  Program  Acquisition  Unit  Cost  (PAUC)  and  Average  Procurement  Unit  Cost 
(APUC). 

A.  NAVY’S  MINEHUNTING  MISSION 

Mine  Countenneasures  (MCM)  can  be  divided  into  two  types:  passive  and  active. 
Passive  countermeasures  involve  reducing  a  ship's  acoustic  and  magnetic  signature  to 
prevent  mines  from  detecting  it.  Active  countenneasures  involve  minesweeping  or 
minehunting.  Minesweeping  is  the  clearing  of  a  pre-defined  area,  whereas  minehunting 
involves  systematic  detection  and  elimination  of  mines  one  at  a  time. 

MCM  techniques  the  Navy  currently  uses  include: 

•  Degaussing  to  reduce  the  ship's  effect  on  the  Earth's  magnetic  field,  thereby 
reducing  the  possibility  of  detection  by  mines 

•  Detection  of  mines  in  shallow  and  deep  waters  using  unmanned  vehicles  towing 
sensors,  such  as  the  remote  minehunting  system  (RMS) 

•  Remote  detonation  of  mines  using  a  standard  mechanical  minesweep  towed 
behind  a  ship  or  helicopter  which  emits  a  ship-like  magnetic  and/or  acoustic 
signal 

MCM  remains  an  essential  naval  mission  when  operating  in  the  littorals  and  is  of 
critical  importance  in  “choke  points,”  such  as  the  Straits  of  Hormuz  or  the  approaches  to 
most  US  commercial  ports.  Accordingly,  the  Navy  continues  to  invest  in  new  and 
increasingly  complex  systems  to  identify,  detect,  and  destroy  sea  mines. 

1.  System  Description 

The  RMS  will  be  an  organic,  off-board  system  that  will  be  launched,  operated,  and 
recovered  from  a  host  surface  ship  and  will  employ  mine  reconnaissance  sensors  that  are 
intended  to  locate  and  identify  mine-like  objects.  Destruction  of  mines,  if  appropriate, 
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would  depend  on  other  non-RMS  systems.  The  AN/WLD-1(V)1  RMS  consists  of  a  set  of 
five  major  subsystems:  (1)  a  Remote  Minehunting  Vehicle  (RMV),  (2)  a  Variable  Depth 
Sensor  (VDS),  (3)  a  Data  Link  Subsystem  (DLS),  (4)  a  Remote  Minehunting  Functional 
Segment  (RMFS),  and  (5)  a  Launch  and  Recovery  Subsystem  (L&RS).  Figure  1 
illustrates  these  five  subsystems. 


Figure  1.  The  Five  Major  Subsystems  of  the  Remote  Minehunting  System 


The  RMV  is  an  unmanned,  high-endurance,  radio-controlled,  semi-submersible 
vessel  that  will  conduct  off-board  mine  reconnaissance  at  extended  ranges  from  the  host 
LCS.  The  only  part  of  the  unmanned  semi-submersible  vehicle  visible  above  the 
waterline  is  the  antenna  mast,  equipped  with  a  video  camera  and  snorkel.  The  RMV 
subsystem  transports  the  sensors,  processors,  and  data  link  equipment  to  the  operations 
area  where  mine  reconnaissance  data  will  be  collected,  recorded,  and  transmitted  via  a 
data  link  to  the  LCS.  The  RMV  tows  the  VDS. 

The  VDS  is  an  actively  controlled  towed  body  (AQS-20A),  providing  a  stable 
platform  for  the  mine  reconnaissance  sensors  and  capable  of  both  depth-specified  and 
altitude-specified  operation.  The  VDS  houses  acoustic  sensor  subsystems  for  detection, 
classification,  and  localization  of  bottom  and  moored  mines  and  an  electro-optic  sensor 
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for  identification  of  bottom  and  moored  mines.  The  AQS-20A  was  originally  developed 
for,  and  is  used  by,  the  Navy’s  MCM  helicopters. 

The  DLS  consists  of  communications  equipment  and  software  that  provides  the 
Line-Of-Sight  and  Over-The-Horizon  radio  telemetry  functions  between  the  RMV  and 
LCS. 

The  RMFS  provides  the  capability  to  command  and  monitor  the  RMV;  receive, 
process,  and  display  mission  data;  conduct  post-mission  analysis;  monitor  performance, 
fault  detection,  and  fault  location;  and  provide  network  communications  to  the  DLS. 

The  L&RS  is  composed  of  equipment  located  on  the  host  ship  designed  to  safely 
launch  the  RMV  into  the  water,  and  then  to  recover  the  RMV.  In  addition,  the  L&RS  will 
host  the  mission  support  equipment,  which  is  composed  of  a  facility  and  the  necessary 
equipment  to  provide  tum-around  and  component  repair  or  replacement  for  the  RMV, 
VDS,  and  DLS  subsystems. 

2.  Timeline  of  Major  Events 

Figure  2  depicts  the  timeline  of  major  events  of  the  RMS  program  from  Milestone 
II  in  1999  through  the  Nunn-McCurdy  breach  in  late  2009.  The  graphic  is  intended  to 
portray  the  Navy’s  program  management,  the  major  milestones  and  designation 
decisions,  contractual  actions  including  reviews  and  deliveries  of  LRIP  units,  and 
development  and  operational  test  events. 

Note  that  the  test  events  are  annotated  with  two  metrics:  availability  and  reliability. 
The  operational  availability  (A0)  is  a  key  perfonnance  parameter,  and  the  mean  time 
between  operational  mission  failures  (MTBOMF)  is  a  key  system  attribute.  For  reference, 
the  threshold  values  for  A0  and  MTBOMF  are  80  percent  and  150  hours,  respectively. 
The  timing  of  these  tests  in  relation  to  LRIP  decisions  is  one  of  the  major  topics  of  this 
analysis. 
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RMV  removed  from  ASW 
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half  (Jan) 


Nunn-McCurdy 
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2000 


2001 


2002 


2003 


2004 


2005 
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;  2007 


2008 


2009 


2010 


EDM  Contract 

Award  (Dec) 

SQT  (OT-IIA) 

OA  (OT-IIB) 

DT2C  (DT-IIC) 
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DT2E  (DT -ME) 

_ 
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A0  =  58% 

— 

A0  =  97% 

A0  =  90% 

A0  =  93% 

A0  =  72% 

A0  =  61% 

MTBOMF  =  22 

MTBOMF  =  76 

MTBOMF  =46 

MTBOMF  =58 

MTBOMF  =42 

MTBOMF  =12 

(Apr) 

(Aug) 

(Sep) 

(Feb) 

(Jun) 

(Sep) 

Critical  Design 

LRIP  Contract 

Review  (Dec) 

Award  (Sep) 

DT2F  (DT-IIF) 
A0  =  46% 
MTBOMF  =92 
(Jul) 


Production  Readiness 
Review  (Nov) 


BOA  Award 
(Jan) 


Preliminary  Design 
Review  (Oct) 


#1  Apr  07 

LRIP  Deliveries 
#4  Jul  08 

#7  Mar  09 

#2  Sep  07 

#5  Nov  08 

#8  TBD 

#3  Dec  07 

#6  Dec  08 

Figure  2.  Timeline  of  Major  Events 


II.  COST  GROWTH  AND  NUNN-MCCURDY  BREACH 


As  shown  in  Figure  2,  RMS  became  an  ACAT  I  program  in  August  2006.  The  program 
surpassed  the  ACAT  I  Research,  Development,  Test  &  Evaluation  (RDT&E)  cost  threshold  due 
to  cost  growth  associated  with  unanticipated  reliability  issues. 1  In  a  memorandum  dated  3  March 
2006,  the  Navy  submitted  a  reprogramming  request  to  the  Under  Secretary  of  Defense 
Acquisition,  Technology  and  Logistics  (USD(AT&L))  to  transfer  $28  million  into  RDT&E.  In 
the  memo,  the  Assistant  Secretary  of  the  Navy  (Research,  Development  &  Acquisition)  wrote, 
“During  subsequent  system  qualification  tests,  reliability  and  suitability  issues  associated  with 
the  RMS  vehicle  were  uncovered  that  now  require  additional  manpower  and  validation  testing  to 
correct  software  and  hardware  in  the  RMS  vehicle  prior  to  commencing  fonnal  DT/OT 
[Developmental  Testing/Operational  Testing].”2 

After  becoming  an  ACAT  I  program,  no  additional  cost  growth  was  reported  until  the 
December  2009  Selected  Acquisition  Report  (SAR).  At  that  time,  the  Navy  notified  in  a  memo 
to  Congress  of  a  critical  Nunn-McCurdy  breach,  reporting  that  the  PAUC  and  APUC  had  risen 
85.1  percent  and  51.2  percent,  respectively,  since  the  October  2006  Acquisition  Program 
Baseline  (APB).  In  the  December  2009  SAR,  the  program  explained,  “The  breach  in  [PAUC] 
and  [APUC]  was  caused  by  a  reduction  in  production  quantities  and  the  use  of  an  incorrect 
average  unit  cost  as  a  basis  of  estimate  in  the  2006  APB.  An  additional  contributing  factor  to  the 
PAUC  breach  was  the  increase  in  RDT&E  costs  from  the  RGP  [Reliability  Growth  Program].” 
IDA  concurs  with  this  assessment. 

There  are  three  relevant  cost  estimates  for  the  RMS  program:  the  2006  APB,  the  2009 
SAR,  and  the  2009  memo  to  Congress.  All  calculations  for  cost  growth  are  relative  to  the  2006 
APB.  The  2009  SAR  reported  cost  growth  from  the  2006  APB  of  79.5  percent  in  PAUC  and  54.6 
percent  in  APUC.  The  2009  SAR,  however,  did  not  accurately  capture  the  program’s  actual  cost 
growth  because  the  Presidential  Budget  Request  (PBR)  did  not  fully  fund  the  RGP  and  did  not 
delay  production  long  enough  to  implement  it  successfully.  (Note  that  DOD  policy  requires  that 
SAR  estimates  comply  with  the  PBR).  The  Navy’s  2009  memo  to  Congress  was  not  constrained 


1  Delores  M.  Etter,  ASN(RDA),  Memo  to  USD(AT&L),  13  March  2006. 

2  James  E.  Thomsen,  PEO  LMW,  Memo  to  USD(AT&L)  via  ASN(RDA),  3  March  2006. 
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to  match  the  budget  request  and  was  somewhat  higher.  IDA  used  the  estimates  from  the  Navy’s 
Nunn-McCurdy  report  to  Congress  in  its  analysis,  shown  in  Table  1. 


Table  1.  Cost  Growth  from  2009  Nunn-McCurdy  Report  to  Congress 


APB 

10/06 

2009 

SAR 

2009  SAR 

%A 

Navy’s 

Estimate 

Navy’s 

Estimate 

%A 

RDT&E 

418.0 

498.8 

19% 

548.1 

31% 

Procurement 

886.6 

672.4 

660.3 

TOTAL 

1,304.6 

1,171.2 

1,208.4 

EDM* 

2 

2 

2 

Production 

106 

52 

-51% 

52 

-51% 

Total  Qty 

108 

54 

-50% 

54 

-50% 

PAUC 

12.1 

21.7 

79.5% 

22.4 

85.1% 

APUC 

8.4 

12.9 

54.6% 

12.7 

51.2% 

*  Engineering  Design  Model 


III.  PROXIMATE  CAUSES  FOR  COST  GROWTH 

A.  REDUCED  PROCUREMENT  QUANTITY 

The  Littoral  Combat  Ship  (LCS)  is  intended  to  be  a  small,  fast,  maneuverable,  and 
relatively  inexpensive  member  of  the  DD(X)  family  of  ships.  The  ship  will  be  recon  figurablc  for 
different  roles,  including  anti-submarine  warfare,  mine  countermeasures,  anti-surface  warfare, 
intelligence,  surveillance  and  reconnaissance,  homeland  defense,  maritime  intercept,  special 
operations,  and  logistics.  Initially,  the  Navy  intended  to  procure  the  RMS  for  both  the  mine 
countermeasures  and  anti-submarine  warfare  mission  modules.  Recently,  the  Navy  decided  to 
procure  a  different,  more  advanced  system  for  anti-submarine  warfare  modules,  thereby  reducing 
its  planned  procurement  quantity  for  the  RMS  from  106  to  52,  resulting  in  higher  average  unit 
costs  for  the  RMS.  Many  reasons  factored  into  this  decision,  although  according  to  internal  Navy 
briefs  regarding  the  FY  2010  President’s  Budget  decisions,  the  issue  with  the  system’s  reliability 
was  not  among  them.  Figure  3  shows  the  change  in  procurement  quantities  between  the  APB  and 
the  most  recent  Navy  plan. 
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Figure  3.  Procurement  Quantities  by  Fiscal  Year  for  the  APB  and  New  Navy  Plan 

A  secondary  effect  of  the  reduced  procurement  quantities  is  a  change  to  the  procurement 
profile.  The  production  rate  fell  from  an  average  of  10  per  year  down  to  a  maximum  of  four.  The 
change  to  the  procurement  profile  also  introduced  a  five-year  production  gap.  This  gap  is 
explained  in  the  next  section. 

B.  RELIABILITY  ISSUES 

The  RMS  has  yet  to  successfully  achieve  its  availability  and  reliability  requirements.  Due 
to  the  system’s  problems  with  availability  and  reliability,  the  Navy  initiated  an  RGP  in  2009  and 
added  $120  million  in  RDT&E  funding  to  support  it.  In  order  to  fully  execute  the  RGP,  the 
program  will  cease  production  for  five  years. 

C.  INCORRECT  ACQUISITION  PROGRAM  BASELINE  ESTIMATE 

The  program  office  estimate  of  $6.3  million  in  the  original  APB  was  low  compared  to  the 
actual  $8  million  average  unit  cost  of  the  first  two  lots.  Thus,  the  cost  growth  stemming  from  this 
is  actually  a  cost  correction;  the  original  official  estimate  did  not  include  some  cost  data  from  a 
negotiated  contract.  The  contract  costs  and  the  APB  estimates  are  listed  in  Table  2. 
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Table  2.  The  Actual  Costs  and  APB  Estimates  for  the  First  Eight  LRIP  Units 


TY$M 

TY$M 

TY$M 

BY06$M 

BY06$M 

LRIP 

Lot# 

Contract 

(9/05) 

Actual 

Cost 

APB 

(10/06) 

Actual 

Cost 

APB 

(10/06)  A 

1 

1 

6.04 

6.04 

5.86 

6.03 

5.86 

2 

1 

9.48 

9.48 

5.87 

9.47 

5.87 

3 

1 

9.48 

9.48 

6.92 

9.47 

6.92 

4 

2 

7.48 

7.48 

6.93 

7.24 

6.70 

5 

2 

7.48 

7.48 

7.24 

6 

2 

8.48 

8.48 

8.20 

7 

2 

8.48 

8.48 

8.20 

LRIP  8 

3 

Average  Actual  (LRIP  1-7)  = 
10.80 

7.98 

6.34  +1.64 

IV.  ROOT  CAUSES  FOR  COST  GROWTH 


IDA’s  analysis  traced  the  above  proximate  causes  of  cost  growth  to  three  root  causes,  two 
of  which  are  directly  related  to  their  proximate  causes  (procurement  reduction  and  the  baseline 
cost  error)  and  one  that  stems  from  poor  government  oversight,  a  symptom  of  which  is  the 
reliability  issues  mentioned  above. 

A.  CHANGES  IN  PROCUREMENT  QUANTITY 

The  Navy  reduced  its  planned  procurement  quantity  from  106  to  52,  as  it  decided  to 
procure  a  different,  more  advanced  system  for  anti-submarine  warfare  mission  module  on  the 
Littoral  Combat  Ship.  The  decision  was  exogenous  to  the  program;  however,  the  reduced 
procurement  quantity  was  a  significant  root  cause  for  the  higher  average  unit  costs  for  the  RMS. 

The  original  RDT&E  cost  estimate  of  $418  million  is  now  spread  over  half  as  many  units, 
which  also  includes  both  the  production  and  two  Engineering  Development  Model  (EDM)  units. 
This  leads  to  a  PAUC  increase  of  $3.87  million.  The  reduced  procurement  also  truncates  the 
learning  curve  from  106  units  to  52,  increasing  the  PAUC  by  $0.22  million  and  the  APUC  by 
$0.37  million.  A  learning  curve  plot  is  shown  later  (Figure  5).  Additionally,  there  is  an  annual 
fixed  cost  of  $5.28  million  associated  with  the  program  that  needs  to  be  considered  in  this 
truncation.  The  original  program  was  intended  to  run  for  13  years  to  complete  106  units.  If  we 
assume  the  same  production  ramp,  the  program  would  only  require  eight  years  to  complete  52 
units.  This  fixed  cost  spread  among  fewer  units  increases  the  PAUC  by  $0.15  million  and  the 
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APUC  by  $0.17  million.  These  factors  together  contribute  to  an  increase  in  PAUC  by  $4.24 
million  and  an  increase  to  APUC  by  $0.54  million.3 

Furthermore,  the  Navy  lowered  the  planned  production  rate  from  an  average  of  10  per  year 
down  to  a  maximum  of  four,  adding  $0.93  million  to  the  PAUC  and  $0.98  million  to  the  APUC. 
The  reason  for  reducing  the  production  rate  is  not  known  to  us. 

The  combined  effect  of  the  reduced  quantity  and  the  change  to  the  procurement  profile 
increases  the  PAUC  by  $5.17  million,  or  43  percent,  and  the  APUC  by  $1.52  million,  or 
18  percent.  The  cost  effect  of  the  five-year  gap  in  production  is  related  to  the  RGP  and  will  be 
addressed  in  the  next  section. 

B.  POOR  PERFORMANCE  BY  THE  GOVERNMENT 

The  Navy  failed  to  follow  appropriate  acquisition  policies  and  procedures  in  three 
significant  ways.  First,  inadequate  contract  planning,  due  to  schedule  pressure  and  personnel 
shortages,  caused  the  government  to  bear  the  majority  of  the  cost  and  risk  during  the  abnormally 
long,  385-day  undefinitized  period  for  the  2005  Low  Rate  Initial  Production  (LRIP)  letter 
contract.  The  Federal  Acquisition  Regulation  Defense  Supplement  states  that  letter  contracts 
should  be  definitized  within  180  days.4 

Second,  the  Navy  awarded  the  LRIP  contract  based  on  build-to-print  tenns  that  was  Firm 
Fixed  Price  (FFP)  for  the  hardware  and  Cost  Plus  Fixed  Fee  (CPFF)  for  engineering  services, 
without  adequately  reviewing  the  contractor  drawings,  resulting  in  subsequent  engineering 
change  proposals  and  delays  at  government  expense.  The  Program  Manager  in  the  Mine  Warfare 
Office  (PMS  495)  approved  the  total  technical  data  package  submitted  by  Lockheed  Martin  in 
September  2007;  however,  the  program  office  individually  approved  and  signed  less  than  10%  of 
the  drawings  that  composed  the  data  package.5 

Third,  the  immature  design  resulted  in  early  developmental  and  production  units  falling 
short  of  the  system’s  performance  thresholds  for  availability  and  reliability  during  testing. 
Although  the  reliability  issues  became  apparent  as  early  as  2005,  the  program  office  did  not 
sufficiently  address  them  before  awarding  any  of  the  three  LRIP  contracts.  The  effectiveness  of 


3  The  calculation  assumes  a  95  percent  learning  curve,  a  fixed  cost  of  $5.28  million  per  year,  and  a  first  unit  cost 
of  $9.25  million.  These  values  are  consistent  with  the  computation  method  used  by  the  Navy.  All  costs  are 
given  in  2006  base  year  dollars.  The  complete  calculation  is  included  in  Appendix  A. 

4  U.S.  Government  Accountability  Office,  “DEFENSE  CONTRACTING:  Use  of  Undefinitized  Contract  Actions 
Understated  and  Definitization  Time  Frames  Often  Not  Met,”  GAO-07-559,  June  2007. 

5  G.  B.  Saroch,  PEO  LMW,  PMS  495,  Memo  to  Lockheed  Martin  Maritime  Systems  &  Sensors.  18  Sept.  2007. 
CAPT  Paul  Siegrist,  PEO  LMW,  PMS  403.  Meeting,  Navy  Yard,  9  April  2010. 
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the  program’s  test-fix-test  reliability  improvements  can  be  assessed  using  a  Duane  Plot,  as  shown 
in  Figure  4.  The  dashed  red  line  indicates  the  threshold  value  of  150  hours  for  the  system’s 
MTBOMF  as  defined  in  the  Operations  Requirements  Document  (ORD).  The  Navy  states  the 
150  hour  threshold  requirement  is  outdated,  since  it  did  not  consider  the  RMS  role  on  the  Littoral 
Combat  Ship,  and  a  more  recent  assessment  of  requirements  places  the  threshold  value  at  75 
hours,  which  is  represented  by  the  solid  red  line.  The  solid  blue  line  indicates  the  reliability 
growth  of  the  RMS  since  testing  began  in  2005.  There  are  two  key  features  to  note:  first,  the 
system’s  reliability  is  below  the  minimum  requirement  of  75  hours  and,  second,  the  rate  of 
reliability  growth  is  slow.  The  maximum  likelihood  estimate  of  reliability  growth  is  0.23, 
according  to  the  Crow-AMSAA  model,  and  the  90  percent  upper  and  lower  confidence  bounds 
are  0.32  and  0.18,  respectively.  Generally,  the  reliability  improvement  slope  across  almost  all 
reliability  improvement  tests  should  fall  between  0.3  and  0.6,  where  the  lower  end  describes  a 
minimally  effective  test  and  the  higher  end  approaches  the  state  of  the  art  for  reliability 
improvement  activities.6  On  this  standard,  the  RMS  program’s  reliability  growth  since  2005  can 
be  considered  only  marginal  at  best,  especially  during  the  later  period  of  testing.7  Some  in  the 
Navy  might  reason  that  the  Duane  methodology  for  accessing  reliability  growth  is  inaccurate,  but 
the  fact  remains  that  after  five  years,  the  system  has  been  unable  to  achieve  an  even  minimally 
acceptable  level  of  reliability. 

The  $120  million  RGP  was  instituted  as  a  consequence  of  the  reliability  problems  and 
increased  the  PAUC  by  $2.22  million,  and  the  five-year  gap  in  procurement  necessitated  by  the 
RGP  further  increases  the  PAUC  by  $0.49  million  and  the  APUC  by  $0.51  million.  The 
combined  effects  on  cost  growth  due  to  reliability  issues  are  an  increase  in  the  PAUC  of  $2.71 
million,  or  22  percent,  and  an  increase  in  the  APUC  of  $0.5 1  million,  or  6  percent. 


6  NIST/SEMATECH  e-Elcmdbook  of  Statistical  Methods,  http://www.itl.nist.gov/div898/handbook/.  February 
2010. 

7  Although  the  slope  through  all  the  points  is  0.23,  the  improvement  after  the  first  two  points  is  considerably 
lower. 
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Figure  4.  Duane  Plot  of  RMS  Test  Data  Since  2005 

C.  ERRONEOUS  BASELINE  ESTIMATES  FOR  COST 

The  APB  cost  estimates  used  unreasonably  optimistic  production  projections  based  on  an 
erroneous  first  unit  cost.  The  program  based  the  estimate  on  the  average  price  of  only  the 
recurring  hardware  production  for  the  four  LRIP  units  in  Lot  2  (LRIP  4-7).  This  price  did  not 
include  sustaining  hardware  engineering,  software  engineering,  integration  and  testing,  or 
program  management.  However,  the  contract  price  for  those  units  includes  those  elements  and  is 
significantly  higher. 

Figure  5  plots  the  hardware  unit  cost  as  a  function  of  cumulative  quantity  using  a 
95  percent  learning  curve.8  The  plot  shows  the  difference  between  the  APB  assumption  using  the 
incorrect  first  unit  cost  (lower  solid  curve)  and  actual  unit  cost  based  on  the  contract  (upper 
dashed  curve).  In  addition  to  neglecting  the  engineering  and  integration  costs,  the  first  unit  cost 
estimate  also  did  not  account  for  cost  associated  with  some  government  furnished  equipment 
(GFE)  in  early  lots.  The  plot  shows  the  actual  cost  of  LRIP  1  when  accounting  for  the  additional 
$4  million  of  GFE. 

The  unrealistic  hardware  cost  estimate  erroneously  reduced  the  APB  numbers,  causing  the 
current  estimates  to  indicate  cost  growth.  This  effectively  increased  the  PAUC  by  $  1 .68  million, 


8  Note  the  hardware  unit  cost  does  not  include  support  cost,  which  the  Navy  estimates  as  $2.7  million  per  unit. 
Taking  this  additional  cost  into  consideration  would  indicate  that  the  overall  cost  growth  due  to  reduced 
quantities  is  under-reported. 
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or  14  percent,  and  the  APUC  by  $1.74  million,  or  21  percent.  This  omission  was  not  corrected  by 
the  Navy  Acquisition  Executive  when  the  program  was  upgraded  to  ACAT  1C  in  2007. 


Cumulative  Quantity 


Figure  5.  Actual  Contract  Prices  Shown  with  APB-assumed  Learning  Curve  (solid)  and  Corrected 

Curve  (dashed) 


V.  CONCLUSION 


In  December  2009  the  Navy  notified  Congress  of  a  critical  Nunn-McCurdy  breach  in  both 
the  PAUC  and  APUC  thresholds  of  the  RMS  program.  The  Navy  reported  that  the  PAUC  and 
APUC  had  risen  more  than  85  percent  and  50  percent,  respectively,  from  the  APB. 

IDA’s  analysis  traced  this  cost  growth  to  three  root  causes,  two  of  which  are  directly 
related  to  their  proximate  causes  (procurement  reduction  and  a  baseline  cost  error)  and  one  that 
manifested  itself  in  reliability  issues  (government  failure  of  oversight). 

Table  3  apportions  the  cost  growth  to  the  major  drivers  and  their  constituent  components. 
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Table  3.  2009  Nunn-McCurdy  Cost  Growth  Apportioned  among  the  Major  Drivers 


In  $M  BY 
FY  2006 

Current  Navy 

Original  APB  Estimate 

Delta 

%  Growth 

PAUC 

12.1  22.4 

+10.3 

85.1% 

Qty  Reduction  &  Profile  Change 

+  5.17 

43% 

Quantity  Reduction 

+  4.24 

35% 

Reduced  Production  Rate 

+  0.93 

8% 

Reliability  Issues 

+  2.71 

22% 

Reliability  Growth  Program 

+  2.22 

18% 

Five-year  Gap 

+  0.49 

4% 

Unrealistic  unit  cost  estimation 

+  1.68 

14% 

Other 

+  0.74 

6% 

APUC 

8.4  12.7 

+  4.3 

51.2% 

Qty  Reduction  &  Profile  Change 

+  1.52 

18% 

Quantity  Reduction 

+  0.54 

6% 

Reduced  Production  Rate 

+  0.98 

12% 

Reliability  Issues:  Five-year  Gap 

+  0.51 

6% 

Unrealistic  unit  cost  estimation 

+  1.74 

21% 

Other 

+  0.53 

6% 
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APPENDIX  A: 
CALCULATIONS 


The  Navy  reduced  its  planned  procurement  quantity  from  106  to  52,  as  it  decided  to 
procure  a  more  advanced  system  for  the  Littoral  Combat  Ship  (LCS)  Anti-Submarine  Warfare 
modules.  Additionally,  the  Navy  will  require  an  additional  $120  million  in  RDT&E  to  execute 
the  Reliability  Growth  Program  (RGP)  and  will  cease  production  of  the  RMS  for  five  years  to 
implement  it.  This  Appendix  details  the  cost  growth  associated  with  these  decisions. 

All  costs  are  in  2006  base  year  dollars. 


Changes  in  Procurement  Quantity:  First,  the  original  RDT&E  cost  estimate  of  $418 
million  is  now  spread  over  half  as  many  units.  Since  the  program  includes  two  engineering 
design  models  in  addition  to  the  procurement  quantity,  the  total  program  acquisition  fell  from 
108  to  54. 


/  $418M  $418M\ 

A  PAUC  from  RDT&E  =  +  (  — ^ - 108“  )  =  +$3'87M 


The  second  aspect  of  unit  cost  growth  is  related  to  the  learning  curve  being  truncated  at  52 
rather  than  106  units.1  The  standard  learning  curve  is  given  by  the  formula 

Cost  of  Unit  Q  —  TXQP , 

where  T1  is  the  theoretical  first  unit  cost,  Q  is  the  cumulative  quantity,  and  /?  is  the  learning 
curve  exponent  defined  as  In(slope) / ln(2) .  Assuming  that  T1  is  $9.25  million  and  the  learning 
slope  is  95  percent,  the  increase  in  the  unit  cost  because  of  the  learning  slope  truncation  is 

/y  52  T  qP  y  106  qP\ 

A  PAUC  from  learning  curve  =  +  I  1 — - - *  *  )  =  "*"$8-22M 

/ y52  p  qP  yioe^  qP\ 

A  APUC  from  learning  curve  =  +  I  1 — - - — —  j  =  +$0.37M 


1  The  learning  curve  calculation  assumes  that  the  EDMs  are  not  considered  as  part  of  the  learning  curve  since  they 
are  built  by  engineers,  not  technicians. 
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Additionally,  there  is  an  annual  fixed  cost  of  $5.28  million  associated  with  the  program  that 
needs  to  be  considered  in  this  truncation.  The  original  program  was  intended  to  run  for  13  years 
to  complete  106  units.  If  we  assume  the  same  production  ramp,  the  program  would  only  require 
eight  years  to  complete  52  units. 


A  PAUC  from  fixed  cost  =  + 


$5.28M*8  $5.28M*13^ 

54  108  ) 


+$0.15M 


A  APUC  from  fixed  cost  =  + 


'$5.28M*8 
.  52 


$5.28M*13^ 
106  ) 


+$0.17M 


Although  the  original  program  intended  to  complete  106  production  units  in  13  years,  the 
new  plan  will  produce  52  units  over  21  years.  We  will  account  for  the  production  gap  of  five 
years  in  the  next  section,  so  for  the  purpose  of  reduced  production  rate,  we  will  consider  the  52 
units  to  be  produced  over  16  years.  The  relative  increase  due  to  the  fixed  annual  costs  is 


A  PAUC  from  changed  rate 


+ 


'$5.28M*16 
.  54 


$5.28M*13^ 

108  J 


+$0.93M 


A  APUC  from  changed  rate  = 


+ 


'$5.28M*16 
.  52 


$5.28M*13^ 

106  J 


+$0.98M 


Consequently,  the  total  change  in  PAUC  and  APUC  due  to  the  reduced  quantity  is: 

A  PAUC  =  RDT&E  +  Learning  Curve  +  Fixed  Cost  +  Rate  Change 
=  +($3.87  +  $0.22  +  $0.15  +  $0.93) 

=  +$5.17M 

A  APUC  =  Learning  Curve  +  Fixed  Cost  +  Rate  Change 
=  +($0.37  +  $0.17  +  $0.98) 

=  +$1.52M 


Reliability  Issues:  The  problems  in  reliability  affect  the  program  in  two  ways.  First,  an 
increased  investment  in  RDT&E  of  $120  million  and,  second,  the  program  production  is  delayed 
five  years  to  allow  time  to  execute  the  reliability  growth  program.  First  the  additional  $120 
million  is  spread  across  54  units. 


/  $120M\ 

A  PAUC  from  RDT&E  =  +1  ^  j  =  +$2.22M 
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Second,  the  annual  fixed  cost  for  the  program  is  $5.28  million,  so  the  five-year  delay 
results  in  an  increase  in  unit  cost  of: 

$5.28M*5 

A  PAUC  from  delay  =  - — - =  $0.49M 


$5.28M*5 

A  APUC  from  delay  =  - — - =  $0.51M 

Consequently,  the  total  change  in  PAUC  and  APUC  due  to  the  reliability  issues  is: 
A  PAUC  =  RDT&E  + Delay 
=  +($2.22  +  $0.49) 

=  +$2.71M 

A  APUC  =  Delay 

=  +$0.51M 
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APPENDIX  B: 

PROBABILITY  RATIO  SEQUENTIAL  TESTING 


The  standard  Probability  Ratio  Sequential  Test  (PRST)  should  be  applied  when  a 
sequential  test  with  normal  (10  percent  to  20  percent)  producer’s  and  consumer’s  risk  is 
desired.  Consumer’s  risk  (P)  is  the  probability  of  accepting  equipment  with  a  true  mean- 
time-between-failure  (MTBF)  equal  to  the  lower  test  MTBF  (9i).  The  probability  of 
accepting  equipment  with  a  true  MTBF  less  than  9i  is  less  than  p.  Producer’s  risk  (a)  is 
the  probability  of  rejecting  equipment  which  has  a  true  MTBF  equal  to  the  upper  test 
MTBF  (9o).  The  probability  of  rejecting  equipment  with  a  true  MTBF  greater  than  9ois 
less  than  a. 

The  sequential  test  is  based  on  the  assumption  that  the  underlying  distribution  of 
times-between-failures  is  exponential.  PRST  plans  will  accept  material  with  a  high 
MTBF  or  reject  material  with  a  very  low  MTBF  more  quickly  than  fixed-duration  test 
plans  having  similar  risks  and  discrimination  ratios,  where  the  discrimination  ratio  is 
defined  as  d  =9o/9i.  Total  test  time  may  vary  significantly;  therefore,  program  cost  and 
schedule  must  be  planned  to  truncation. 

Using  the  PRST  technique  shown  in  Figure  B-l,  the  Navy  had  sufficient 
information  after  the  first  system  qualifying  test  (SQT  I)  in  2995  to  conclude  that  the 
initial  design  did  not  meet  the  reliability  requirements.  The  plot  assumes  a  consumer  and 
producer  risk  of  19  percent.  The  lower  test  MTBOMF  is  69  hours  and  the  upper  test 
MTBOMF  is  159  hours.1  The  data  points  are  labeled  with  test  names  and  dates.  Because 
reliability  was  clearly  unsatisfactory,  the  program  should  not  have  proceeded  to  LRIP 
without  a  robust  RGP  in  place.  Nevertheless,  despite  the  system’s  poor  test  performance 
and  minimal  reliability  growth,  the  Navy  committed  to  an  initial  and  two  subsequent 
LRIP  buys.  The  Navy  procured  three  units  in  July  2995,  four  units  in  September  2996, 
and  one  unit  in  April  2998,  at  a  total  cost  of  $193.7  million. 


1  These  levels  were  chosen  for  illustrative  purposes,  but  for  any  reasonable  combination  of  values,  the 
data  indicate  the  system  should  have  been  rejected,  or,  at  best,  testing  should  have  continued. 
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Number  of  Failures 


Figure  B-1.  Probability  Ratio  Sequential  Test  Results 
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APPENDIX  C: 
ACRONYMS 


APB 

APUC 

ASN(RD&A) 

CPFF 

DLS 

DT 

FFP 

GFE 

IDA 

L&RS 

LCS 

LRIP 

M 

MCM 

MTBOMF 

OT 

PAUC 

RGP 

RMFS 

RMS 

RMV 

SAR 

USD(AT&L) 

YDS 


Acquisition  Program  Baseline 
Average  Procurement  Unit  Cost 

Assistant  Secretary  Navy  Research,  Development  &  Acquisition 

Cost  Plus  Fixed  Fee 

Data  Link  Subsystem 

Development  Test 

Firm  Fixed  Price 

Government  Furnished  Equipment 
Institute  for  Defense  Analyses 
Launch  and  Recovery  Subsystem 
Littoral  Combat  Ship 
Low  Rate  Initial  Production 
million 

Mine  Countermeasures 

Mean  Time  Between  Operational  Mission  Failures 

Operational  Test 

Program  Acquisition  Unit  Cost 

Reliability  Growth  Program 

Remote  Minehunting  Functional  Segment 

Remote  Minehunting  System 

Remote  Minehunting  Vehicle 

Selected  Acquisition  Report 

Under  Secretary  of  Defense  Acquisition,  Technology  and  Logistics 
Variable  Depth  Sensor 
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